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ABSTRACT 

The use of visual aids is expected to have a positive effect on students' learning. However, not all visual aids work equally 
well. A recent meta-analytic research which examined 42 studies has found that the use of animated visuals does not 
facilitate learning (Anglin, Vaez & Cunnincham, 2004). The failure of visual aids can be attributed to factors such as 
violating the congruence and apprehension principle, lacking appropriate control functions, and mediation by 
individual characteristics. Across the board, researchers note the lack of appropriate user-control functions as the key 
factor. 

Interactivity, or the user-control mechanisms of animation, is pointed out as a factor for users to better apprehend the 
knowledge which the visual aid aims to deliver. The functions of stopping, starting and replaying allow students to re¬ 
inspect the animation. Consequently, students can focus on specific parts and actions of the visual aid. In addition, 
animations that allow users to control close-ups, zooming, and speed are more likely to facilitate perception and 
comprehension (Tversky et al, 2002). This study focuses on creating an animated visual aid with appropriate self- 
controlled functions to better help students' learning of introductory statistics. We consider different levels of interaction 
for mediating students' individual differences. 

Keywords: Animated Visual Aid, Statistics Education, Instructional Design, User-control Function. 


INTRODUCTION 

Proposal 

The teaching and learning of statistics has pervaded all 
levels of education, including post-secondary, college- 
level and graduate-level curriculums (Garfield & Ahlgren, 
1988). For college-and graduate-level students, statistics 
has become a requisite course for a wide range of fields. 
This level of education requires enhanced cultivation of 
quantitative and logical thinking, and additionally 
provides skills necessary for future employment choices. 
College statistics courses are expected to offer students 
tools for trimming the mass of information, ordering the 
messiness, separating sense from nonsense, and 
selecting the relevant few from the irrelevant many in real 
world data (Ben-Zvi, 2000). However, students face many 
challenges in statistics courses. Researchers have found 
that students are not capable of obtaining an adequate 


understanding of basic statistics concepts. Also, students 
are not able to solve applied problems (Garfield, 1995). 
Studies show that students often respond to statistical 
problems by falling into a "number crunching" mode. 
That is, students plug in quantities into a computational 
formula or procedure without forming an internal 
interpretation of the problem (Garfield & Ahlgren, 1988). 
The development of computer technology and the 
Internet provides solutions for college instructors to 
improve students' understanding of statistics. The visual 
aid, one powerful type of technological tool, has a high 
potential to overcome students' learning obstacles in 
statistics and decrease the learning curve. The term visual 
aid refers to a visual representation or a visual display that 
facilitates students' learning. Visual representations and 
visual displays are used interchangeably in the literature 
to characterize displays that represent objects. 
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concepts, and their relations by using symbols and their 
spatial arrangement (Vekiri, 2002). Visual aids are 
frequently integrated into the classroom setting to 
support learning and to make the learning process more 
effective. The purpose of graphical displays is to provide 
the viewer a visual means of processing information 
(Segenchuk, 1997). 

Since 1980, the visual aid has been viewed as an 
innovative strategy in education, especially for math or 
science related subjects. Different types of visual aids 
have been developed to enhance students' concepts 
formation. In computer science, for example, the 
impetus for visual aids comes from the abstractness of 
the concepts in the field. To make these concepts more 
concrete, graphical representations are suggested to 
help one to better understand how they work (Naps et at., 
2003). Also, in physics and chemistry classes, it is very 
common for teachers to integrate visualizations to show 
the molecular structure. To give students a clear 
understanding of molecular modeling, instructors can 
show a molecular simulation or engage students in 
creating a molecular simulation in the class (Allen, 2007). 
In a biology class, the instructor might use a simulated 
frog dissection as a preparation for an actual dissection 
or as a substitute for the dissection (Kulik, 2002). Medical 
science is also a field which relies on visual aids; 
simulation is used to train medical practitioners to 
diagnose patients (Savoldelli et at., 2005). The above 
examples show the possibilities of integrating visual aids 
into educational use. 

One popular way to organize visual displays is to 
categorize them into static or animated visuals, based on 
whether their graphical components can change with 
time or not. Static visuals are those in which the graphical 
elements being displayed can not change with time 
(Vekiri, 2002). Researchers began to explore the 
educational value of static visuals in the 1930's. Many 
functional frameworks have been presented for 
categorizing different type of static visuals. An example of 
a static visual is University of Washington's Digital 
Anatomist Project, which includes digital atlases to teach 
students human organs (Structural Informatics Group, 


2007). According to Segenchuk (1997), the static visuals 
are capable of drawing an analogy to another domain 
of knowledge, serving as an intermediate representation 
to bridge the learners' prior knowledge and new 
knowledge, and making salient the important 
relationships within a concept. 

Animated visuals are those in which the graphical 
elements being displayed can change with time (Vekiri, 
2002). The study of animated visual aids began in the 
1920's when the instructional film became popular. But 
with the progress of technology development such as 
JAVA and Flash, the animated visual aids now have been 
expanded from films to computer-based or internet- 
based software programs. This type of visual aid is 
increasing in educational materials across a range of 
subject disciplines and levels of study. Animations and 
simulations are examples of animated visuals. Animated 
visuals bring three attributes to an instructional setting: 
visualization, motion, and trajectory (trajectory refers to 
the direction of the path of travel of an animated object) 
(Rieber, 1990). Since animated visual aids create a more 
dynamic and interactive environment for learners than 
static visual aids, they are viewed as more powerful in 
helping students to construct mental schema. 
Researchers or commercial units in the statistics 
education field have designed large amounts of online 
visual aids. In the 80's, tools with text-based interfaces 
such as Minitab and Graphing Calculator were popularly 
used to carry out the visualization and simulation strategy 
in the statistics classroom. Teachers used them to 
simulate distribution (Dambolena, 1986; Dambolena, 
1986), Bayesian method (Albert, 1993) and the central 
limit theorem (Gordon, 1987). In the 1990's, educational 
visual aids with graphic-based interfaces were 
developed. Among these tools, ELASTIC (Environment for 
learning Abstract Statistical Thinking) was the most 
popular visual aid in statistics education (Rosebery & 
Rubin, 1989, 1990). In the 2000's, animated visual aid with 
internet-based Interface emerged. JAVA applets and 
flash applications continue to be the most popular tools 
among Internet-based tools. JAVA applets are computer 
applications designed for the internet. Applets are 
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platform-independent, meaning that they can run on 
any operation system that has a JAVA Virtual Machine to 
translate applet byte codes to appropriate platform- 
dependent instructions. The advantages of JAVA applets 
include its better performance in speed, interoperability, 
user interaction, portability, network computing and 
development and maintenance (Kamthan, 1999). Some 
noted JAVA applets for statistics education are: Rice 
Virtual Labs in statistics, Elementary Statiscs JAVA applets, 
Statlets, StatCrunch, Statiscope, PsychStat, BusinessStat, 
Probability by Surprise, Web Interface for Statistics 
Education, CUWU Stats, PSOL, StatLab, Virtual Labs in 
Probability & Statistics, JavaStat, Vestac, JSci and 
CyberStats (Dinov et al., 2006). Flash is the most recent 
program for visual aid display in statistics education. One 
example is ConStatS, developed by Tufts University. 
ConStatS consists of 12 modules grounded into five 
distinct parts: representing data, probability, sampling, 
inference, and experiments. Constats uses Flash to 
develop the "point and click" graphic interface. Pull 
down menus are used to access datasets, exercises, 
experiments, anddifferenttopics. 

The use of visual aids is expected to have a positive effect 
on students' learning. Plowever, not all visual aids work 
equally well. In real world practice, instructors are often 
unsuccessful in using them. A recent meta-analytic 
research which examined 42 studies has found that the 
use of animated visuals does not facilitate learning 
(Anglin, Vaez & Cunnincham, 2004). The failure of visual 
aids can be attributed to factors such as violating the 
congruence and apprehension principle, lacking 
appropriate control functions, and mediation by 
individual characteristics. Across the board, researchers 
note the lack of appropriate user-control functions as the 
key factor. 

Interactivity, or the user-control mechanisms of 
animation, is pointed out as a factor for users to better 
apprehend the knowledge which the visual aid aims to 
deliver. The functions of stopping, starting and replaying 
allow students to re-inspect the animation. Consequently, 
students can focus on specific parts and actions of the 
visual aid. In addition, animations that allow users to 


control close-ups, zooming, and speed are more likely to 
facilitate perception and comprehension (Tversky et al., 
2002 ). 

Due to technological limitations, early researches on 
animated visual aids focused on uncontrollable 
animations. It is in the past five years that researchers 
begin to focus on the use of user-control animation in 
learning. Fiowever, the current findings are not yet 
inclusive. Some argued that the user-control functions 
are significantly helpful to learners (Mayer, 2001) but 
some do not. (Boucheix & Schneider, 2008) presented 
significant experimental evidence showing that 
providing user-control can not guarantee a positive 
learning result. This is because the user may not be able to 
respond to the user-control functions due to his prior 
knowledge or information management skills. In other 
words, the design of the user-control function needs to 
match users' ability or skills, or the use of animation may 
not succeed at all, more research needs to be done. 

This study focuses on creating an animated visual aid with 
appropriate self-controlled functions to better help 
students' learning. The authors consider different levels of 
interaction for mediating students' individual differences. 
The research questions are as follows: 

• Fiow do we establish user-controlled functions in the 
animated visual aid? 

• Fiow does the use of animated visual aids improve 
students' academic performance and confidence 
in statistics? 

Flopefully this study will provide an example of how to 
design a user-controlled animated visual aid, as well as 
broaden the understanding of how the animated visual 
aid impact on learning. Also, this research could serve as 
a reference for future statistics curriculum designers, 
textbook publishers or instructors to design or implement 
the curriculum or courses using visual aids. 

System Design and Development 
Content Selection 

The unit which the authors selected to develop an 
animated visual aid for is the principles of hypothesis 
testing. The purpose of our design is to help students learn 
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to set up a hypothesis test and to interpret the results. Most 
students are able to follow the basic procedures for 
testing hypotheses. However, even after discussing what 
Type I and II errors signify, students inevitably ask, "but 
where do you get alpha(a)?" (Smith, 2003) Students' 
difficulties in grasping the underlying principles of 
hypothesis testing strongly reduce their ability to interpret 
the testing results. Though these are important concepts 
in fundamental statistics, few studies focus on cultivating 
of students' understanding of hypotheses testing. A visual 
aid was designed and developed to uncover students' 
difficulties and test whether the aid could improve their 
understanding. Through the learning of this unit, students 
are expected to distinguish the null hypothesis from the 
alternative hypothesis, and apply the concept of alpha, 
beta and p-value to make decisions in the significance 
testing. The specific objectives which students need to 
achieve include: 

• The student is able to set up the null hypothesis as well 
as the alternative hypothesis using real-world 
problems. 

• The student is able to understand the errors (type I and 
type II error) which could be made in the testing 
process. 

• The student is able to calculate a, p, and p-value in 
real-world problems with different population shapes. 

• The student is able to make a decision to the real- 
world problem based on the number of a, p, and p- 
value. 

In the traditional classroom, the instructor teaches above 
concepts by using the textbook and blackboard. The 
following is one sample page of this unit from a textbook 
(Figure 1) 

With the use of the textbook, there is no way for students to 
interact with the numbers. All students can do is to 
passively read the definition and examples on the 
textbook. As a result, the effectiveness of it is very limited. 
This is why an animated visual aid is necessary here. The 
author's design of the animated visual aid is based on this 
introductory statistics textbook (Vaughn, 200X). The 
animated visual aid is expected to deliver the same 
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Figure 1. The exemplary page of traditional textbook for 
statistical language of hypothesis testing 


contents as the textbook will, and the following section will 
elaborate it. 


System interface design 

The visual aid program is created by using Adobe Flash 
CS3 Professional and Action Script 2.0. It is designed as a 
480px * 560px window. The small screen size of the visual 
aid makes it possible for the instructor and students to put 
it on any mobile device. But in this particular study, the 
laptop is the main device used(Figure 2). 

The user-controlled mechanism is built inside the visual 
aid program. That is, users are allowed to have different 
levels of interactions based on their needs and 
preferences. Two different modes are created for users to 
interact with: the viewing mode and the practicing 
mode. The users are free to switch between two modes 
for learning. 

The viewing mode allows students to watch the animated 
procedure of calculating a, p, and p-value. Students are 
allowed to select a HO distribution, draw a random 
voucher from the bag, and set up a decision rule on the 
left-side area of the window. After the student click on the 
"viewing calculating" button, the processes of 
calculating a, p, and p-value will be shown on the right- 
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side of the window with graphs and boxes. Students are 
allowed to stop, start and replay any animation for re¬ 
inspection. The Figure 2 shows the viewing mode 
interface. 

Figure 2 shows the calculation procedure of a one-sided 
testing. The left-side area on the window allows students 
to have different inputs. In this case, the HO distribution is 
selected as a "left-tailed" distribution; the voucher drawn 
is set up as "70", and the decision rule is set up as "Reject 
HO if value <= 30". The right-side area of the window 
shows the animation of calculation for a, p, and p-value. 
Three figures are used to illustrate the calculation. The top 
one shows the calculating procedure for a. The black 
boxes represent the voucher values in the HO bag, and 
the white boxes represent those vouchers which fall into 
the rejection region. In this case, while calculating a, 
there are 20 boxes (vouchers) in the bag, and only 4 fall in 
the rejection region. As a result, a equals to four out of 
twenty (4/20). Similar calculation processes are animated 
on the middle and bottom figure for p and p-value. After 
the calculations are finished, the testing conclusion will 
be shown as texts on the bottom of the window. In this 
case, the program tells the students "since p-value>a, 
we fail to reject HO." 

Only providing students the opportunities to re-inspect 


the animated procedure is not enough. In the visual aid 
program, the authors create another mode for students 
to have more advance interactions. That is, the 
practicing mode. Similar to the viewing mode, the 
practicing mode also allows students to select the HO 
distribution, to draw a voucher from the bag, and to set 
up a decision rule. Flowever, this time, students need to 
calculate the a, p, and p-value by themselves. There are 
totally 7 questions for students to solve in each round, and 
they need to use the graphs provided to do the 
calculation. The program will no longer show them any 
animation here. Students are asked to select the correct 
distribution by using the drop-down menu for each 
question and click on the black boxes on the graphs for 
doing calculating. After finishing calculating, students 
have to decide if the HO hypothesis is rejected or 
accepted by selecting one item on the drop-down 
menu. When all questions are done, students can click on 
the "check my answer" button to check if they get all 
answers right. The program will show students if they get 
everything correct. The students can keep on trying until 
they get all answers right. 

Figure 3 gives an example of how the student practices 
questions in the practicing mode. Similar with the viewing 
mode, the left-sided area on the window allows the 
students to have different inputs. In this case, the "left¬ 
tailed" distribution is selected as the HO distribution, the 



Figure 3. System Interface for the practicing mode 
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voucher value is set up as "20," and the decision rule is set 
upas "reject HO if value <= 30." After the student clicks on 
the "do calculation" button, 7 questions will be shown on 
the right-side of the window. The first two questions ask 
students to select a distribution for a and click on the 
black boxes to mark those which tall in the rejection 
region. In this case, the student selects the "HO 
distribution" for a, and he clicks on the left two boxes to 
show the rejection region. As a result, the student's answer 
of a is two out of twenty (2/20). The calculation practice of 
P and p-value are similar with that of a. For the value of p, 
in this case, "HI distribution" is selected and the right three 
boxes are marked white. The student's answer to p is three 
out of twenty (3/20). For the p-value, in this case, "HO 
distribution" is selected and the left three boxes are 
marked white. The student's answer to p is three out of 
twenty (3/20). After comparing a and p-value, the student 
chooses to accept HO. However, while the student 
checks the answers, the program tells the student he gets 
three questions wrong: #3, #5, and #8. That is, the 
student has wrong answers to a, p, and the final decision. 
The student is allowed to redo this practice until he gets 
everything right. Or he may go back to the viewing mode 
to re-inspectthe animation of calculation process. 

There is no restriction for students to switch between the 
two different modes. Consequently, students could play 
in the viewing mode, go ahead to the practice mode, 
and then go back to the viewing mode again. 

System Evaluation 
Participants 

The animated visual aid was put into an undergraduate 
introductory course for examining its effectiveness. 18 
students are voluntarily joined in this study. These students 
came from different college and department with 
different majors, however, they are all novice learners of 
statistics and this course is their first course during their 
college life. 

Pre- and post-tests 

The pre- and post-tests are two paralleled tests in which 
questions are with the same content and difficulty. There 
are 12 multiple-choice questions on each test. For 


examining students' academic improvement in detail, 
they decompose the knowledge covered in this unit into 
3 levels: the remembering-/understanding-level 
knowledge (3 questions), the applying-level knowledge (7 
questions), and the analyzing-level knowledge (2 
questions). For each question, there is a following 
question asking about student's confidence level, which 
ranges from 1 (very unconfident) to 4 (very confident). The 
reliability forthe pre-test and post-test is ,73and .60. 
Procedure 

At the beginning of this study, all participants were given a 
3-session lecture with the use of the textbook for learning 
statistical language and hypothesis testing. After finishing 
the lecture, students were asked to have a pre-test to 
evaluate their current knowledge of this particular unit. 
Then participants were asked to play with the animated 
visual aids. There was no time limitation for playing with 
the animated visual aids. Participants were free to stay in 
any mode of the animated visual aid and they are 
allowed to switch between the two modes. Most 
participants finished the visual aid manipulation in two 
hours. The post-test was assigned to these participants 
while participants finished up playing with the animated 
visual aid. 

Findings 

The scores on the pre- and post-test provided an 
assessment of the participant's general understanding 
about statistical language and hypothesis testing. The 
tests forthe pre- and post-test scores are shown in Table 1 
andTable2. 



Pre-test 

Post-test 

Paired T-test 


Mean 

Sd. 

Mean 

Sd. 

't' 

Sig. 

(2-tailed) 

Remembering- 

/understanding 
level knowledge 

2.72 

.57 

2.67 

.49 

.0369 

.717 

Applying- 
level knowledge 

4.06 

2.21 

5.17 

1.50 

-2.938 

0.009 ** 

Analyzing- 
level knowledge 

1.72 

.46 

1.89 

.32 

-1.144 

.269 


* Maximum possible score on remembering-/understand-leveling knowledge is 3 

* Maximum possible score on applying-level knowledge is 7 

* Maximum possible score on analyzing-level knowledge is 2 


Table 1. Tests on students' academic performance 
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Pre-test 

Post-test 

Paired T-test 


Mean 

Sd. 

Mean 

Sd. 

‘t‘ 

Sig. 

(2-tailed) 

Remembering- 

/understanding 
level knowledge 

3.75 

.28 

3.89 

.26 

-.167 

.110 

Applying- 
level knowledge 

2.80 

.82 

3.26 

.55 

-2.57 

0.020** 

Analyzing- 
level knowledge 

3.09 

1.00 

3.72 

.39 

-3.00 

.008** 


* Maximum possible score on students' confidence is 4 


Table 2. Tests on students' academic confidence 
It can be seen from Table 1 and Table 2 that students' 
academic performance and confidence in the apply- 
level knowledge is significantly improved after the using 
the animated visual aid (t=-2.938, p=.009; t=-2.57, 
p=.020). The findings indicate that the animated visual 
aid contributes to students' learning of statistical 
language and hypothesis testing. 

Discussion 

The animated visual aid has a long history used in 
education including the statistics field. However, from the 
literature review, it is found that the helpfulness of the 
animated visual aid is over-expected. Many studies 
claimed that the animated visual aid does not contribute 
to learning. One of the important factors of the failure of 
animated visual aid is its lack of appropriate user-control 
functions. That is, animated visual aids can not work out if 
it does not allow users to stop, start and replay an 
animation for re-inspection. 

The purpose of this study is to develop an animated visual 
aid which the users can have fully control. Therefore the 
researchers can further explore if the user-controlled 
visual aid helps learning and how it helps learning. They 
selected the principles of hypothesis testing as the unit for 
developing the animated visual aid, and the learning 
targets and content are all clearly defined before they 
develop it. Our animated visual aid is finally divided into 
two modes, the viewing mode and the practicing mode, 
which provide different level of interactions but the users 
can fully control any of the two modes. Also, users can 
freely switch between the two modes for figuring out 
concepts. 


A small-scale system evaluation was conducted to see 
the performance of this animated visual aid. The system 
evaluation is designed as a pre- and post-test experiment 
with 18 voluntary undergraduate students. The results 
show that students' applying-level knowledge is 
significantly improved after the use of the animated 
visual aid. The applying-level knowledge here refers to 
make use of information in a context different from the 
one which it was learning. In other words, the applying- 
level knowledge means the ability to use strategy, 
concepts, principles and theories in new situations. It 
relates to learners' ability to compute, demonstrate and 
manipulate. In the principles of hypothesis testing unit, the 
applying-level knowledge means the students are able 
to set up the null/alternative hypothesis, calculate alpha, 
beta, and p-value with different population shape, and 
make a decision to the real-world problem. 

The study shows that the use of user-controlled visual aid 
can significantly improve students' applying ability. Here 
they attach some applying-level questions which we use 
to evaluate students' knowledge. The following question is 
test question #13 for testing students' ability of 
calculating p-value. 

QUESTION 13. There are two identical looking wallets - A 
andB and here are descriptions of the contents: 

Wallet A: One $1 bills. Five $5 bills. Ten $10 bills 
WalletB: Eight $1 bills. Seven $5 bills. One $10 bills 
A wallet is picked at random and exactly one bill is 
drawn at random from that wallet and shown to you. 
Based on the observation, you must decide which 
wallet the bill was drawn from. 

Consider the following hypothesis. 

HO: The bill is drawn from wallet A 

HI: The bill is drawn from wallet B 

Consider the decision rule: Reject HO if the bill drawn 

is <=5, otherwise accept HO.. 

For the given decision rule, if the bill picked up is $1, 
compute the p-value 

Before playing with the animated visual aid, only 7 
students got this question correct. After playing with the 
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animated visual aid, 12 students got this question correct. 
The following another question testing students' ability of 
calculating beta, the type II error. 

QUESTION 15. A box contains many coins and all of them 
look identical. However only some coins are fair - when 
you toss a fair coin, the probability of getting heads is 
1/2. The other coins are biased and assume that when a 
biased coin is tossed, the probability of getting heads is 
2/3. They are given one coin from the box and asked to 
determine if it is a fair coin, by tossing the coin exactly 
once and basing our decision on the outcome. 

Consider the null hypothesis, HO: The tossed coin is a 

fair coin. 

Consider the decision rule: Reject HO if we get heads. 

Calculate (3, the chance of making Type II error. 
Before playing with the animated visual aid, only 5 
students got this question correct. After playing with the 
animated visual aid, 9 students got this question correct. 
Students' scores significantly arise in almost every item of 
the applying-level questions. 

Students' improvement not only reveals in their 
academic grades. Their confidence for solving applying- 
level question is also significantly enhanced. For 
applying-level knowledge, the students' average 
confidence level arises over 3 after playing with the visual 
aid. In this study of 4-point confidence scale, 3 is the 
starting point where students feel confident to their 
answers. In other words, the use of animated visual aid 
leads students from the unconfident status to the 
confident stats while solving the application questions. 
The enhancing of students' confidence in solving statistics 
questions helps cultivating their positive attitude and 
motivation in learning statistics. Consequently it is 
valuable to instructors to integrate user-controlled visual 
aid in statistics course. 

Interestingly, students' improvement is only shown in the 
applying-level knowledge. It seems students' 
remembering, understanding and analyzing level 
knowledge is not significantly improved. This might 
because this particular animated visual aid is directly 
designed for solving applied questions. In the visual aid 


environment, students are free to watch and practice the 
calculation of alpha, beta, and p-value with different kind 
of inputs. However, the remembering and understanding 
level knowledge is not directly delivered from the visual 
aid. So is the analyzing-level knowledge. For improving 
different level of knowledge other than the applying-level 
one, more instructions or associated worksheet might be 
designed and provided. How to optimize the use of 
animated visual aid for covering all level of knowledge 
will be the future goal of this study. 

Conclusion 

This study aims to develop a user-controlled visual aid to 
facilitate students learning statistics and examine how it 
helps students' knowledge and confidence improvement. 
The findings show that our particular animated visual aid 
could significantly improve student's academic 
performance and confidence in applying-level 
knowledge. For future research, we will continue revise 
the interactive functionality of the animated visual aid, 
create more associated worksheet or booklet for 
covering more level of knowledge, and increase our 
sample size to have more stable evidences of the 
strength and limitation of the animated visual aid. 
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